The exit time finite state projection scheme: bounding exit distributions and occupation measures of continuous-time Markov chains
نویسندگان
چکیده
We introduce the exit time finite state projection (ETFSP) scheme, a truncation-based method that yields approximations to the exit distribution and occupation measure associated with the time of exit from a domain (i.e., the time of first passage to the complement of the domain) of a continuous-time Markov chain. We prove that: (i) the computed approximations bound the desired measures from below; (ii) the total variation distances between the approximations and the measures decrease monotonically as states are added to the truncation; and (iii) the scheme converges in the sense that as the truncation tends to the entire state space, the total variation distances tend to zero. Furthermore, we give a computable bound on the total variation distance between the exit distribution and its approximation, and we delineate the cases in which the bound is sharp. To establish the theoretical properties of the ETFSP scheme, we revisit the related finite state projection scheme taking a probabilistic viewpoint and proving the latter’s convergence. We demonstrate the use of the ETFSP scheme by applying it to two biological examples: the computation of the first passage time associated with the expression of a gene, and the fixation times of competing species subject to demographic noise.
منابع مشابه
ANALYSIS OF FINITE BUFFER RENEWAL INPUT QUEUE WITH BALKING AND MARKOVIAN SERVICE PROCESS
This paper presents the analysis of a renewal input finite buffer queue wherein the customers can decide either to join the queue with a probability or balk. The service process is Markovian service process ($MSP$) governed by an underlying $m$-state Markov chain. Employing the supplementary variable and imbedded Markov chain techniques, the steady-state system length distributions at pre...
متن کاملFlow and Pressure Distributions in Short Heat Exchanger Cores with Abrupt Entrance and Exit
The typical installation of a heat exchange device usually involves a flow contraction at the core entrance and a flow expansion at the core exit. Repeated flow Contraction and expansion are experienced in the flow passages of some compact heat exchangers. The latter refers to the flow passages in the plate-fin type with louvered fins or stripped fins and in the tubular type with dimpled-circul...
متن کاملCSRL model checking with closed-form bounding distributions
Continuous Stochastic Logic (CSL) which lets to express real-time probabilistic properties on Continuous-Time Markov Chains (CTMC) has been augmented by reward structures to check also performability measures. Thus Continuous Stochastic Reward Logic (CSRL) defined on Markov Reward Models (MRM) provides a framework to verify performancerelated and as well as dependability-related measures. Proba...
متن کاملStopping rule reversal for finite Markov chains
Consider a finite irreducible Markov chain with transition matrix M = (pij). Fixing a target distribution τ , we study a family of optimal stopping rules from the singleton distributions to τ . We show that this family of rules is dual to a family of (not necessarily optimal) rules on the reverse chain from the singleton distributions to a related distribution α̂ called the τ -contrast distribut...
متن کاملSum of exit times in a series of two metastable states
Abstract. We consider the problem of non degenerate in energy metastable states forming a series in the framework of reversible finite state space Markov chains. We assume that starting from the state at higher energy the system necessarily visits the second one before reaching the stable state. In this framework we give a sharp estimate of the exit time from the metastable state at higher ener...
متن کامل